update vector search docs #18779
Open
update vector search docs #18779
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ti-chi-bot
bot
added
missing-translation-status
ti-chi-bot
bot
removed
the
missing-translation-status
ti-chi-bot
bot
added
size/L
Oreoxmt
approved these changes
Sep 3, 2024
[LGTM Timeline notifier]Timeline:
|
ti-chi-bot
bot
added
size/XL
qiancai
added
the
do-not-merge/hold
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/approve cancel |
ti-chi-bot
bot
added
the
needs-rebase
|
PR needs rebase. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
qiancai
added
v8.4
qiancai
removed
the
needs-rebase
ti-chi-bot
bot
added
size/XXL
breezewish
reviewed
Sep 25, 2024
|
|
||
| > **Warning:** | ||
| > | ||
| > The vector search feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. |
There was a problem hiding this comment.
Suggested change
| > The vector search feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. | |
| > The vector search feature is experimental and some behaviors may change in future versions. It is not recommended that you use it in the production environment. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. |
Remove "This feature might be changed or removed without prior notice." as Vector Search is experimental because of stability issues, not product decision issues. Vector Search will never be removed.
breezewish
reviewed
Sep 25, 2024
|
|
||
| > **Warning:** | ||
| > | ||
| > The vector search feature is in beta. It might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. |
There was a problem hiding this comment.
Suggested change
| > The vector search feature is in beta. It might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. | |
| > The vector search feature is in beta and some behaviors may change in future versions. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. |
EricZequan
reviewed
Sep 26, 2024
| CREATE TABLE foo ( | ||
| id INT PRIMARY KEY, | ||
| data VECTOR(5), | ||
| data64 VECTOR64(10), |
There was a problem hiding this comment.
Suggested change
| data64 VECTOR64(10), |
We do not support this syntax.
Signed-off-by: JaySon-Huang <[email protected]>
|
@qiancai: The following test failed, say
Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here. |
Comment on lines
+127
to
+130
| ## From v7.x to v8.4 or a later version | ||
|
|
||
| Starting from v8.4, the underlying storage format of TiFlash has been updated to support the [vector search](/vector-search-overview.md). Therefore, after the upgrade TiFlash to v8.4 or a later version, in-place downgrading to the original version is not supported. | ||
|
|
There was a problem hiding this comment.
FYI, I've pushed a commit about the tiflash upgrade notice
hfxsd
reviewed
Sep 30, 2024
Comment on lines
+24
to
+26
| TiDB currently supports the following vector search index algorithm: | ||
|
|
||
| - HNSW |
There was a problem hiding this comment.
Suggested change
| TiDB currently supports the following vector search index algorithm: | |
| - HNSW | |
| TiDB currently supports the [HNSW (Hierarchical Navigable Small World)](https://en.wikipedia.org/wiki/Hierarchical_navigable_small_world) vector search index algorithm. |
| - TiFlash nodes must be deployed in your cluster in advance. | ||
| - Vector search indexes cannot be used as primary keys or unique indexes. | ||
| - Vector search indexes can only be created on a single vector column and cannot be combined with other columns (such as integers or strings) to form composite indexes. | ||
| - A distance function must be specified when creating and using vector search indexes (currently, only cosine distance `VEC_COSINE_DISTANCE()` and L2 distance `VEC_L2_DISTANCE()` functions are supported). |
There was a problem hiding this comment.
Suggested change
| - A distance function must be specified when creating and using vector search indexes (currently, only cosine distance `VEC_COSINE_DISTANCE()` and L2 distance `VEC_L2_DISTANCE()` functions are supported). | |
| - A distance function must be specified when creating and using vector search indexes. Currently, only cosine distance `VEC_COSINE_DISTANCE()` and L2 distance `VEC_L2_DISTANCE()` functions are supported. |
| > | ||
| > The vector search feature is experimental. It is not recommended that you use it in the production environment. This feature might be changed or removed without prior notice. If you find a bug, you can report an [issue](https://github.com/pingcap/tidb/issues) on GitHub. | ||
|
|
||
| </CustomContent> | ||
|
|
There was a problem hiding this comment.
Suggested change
| <CustomContent platform="tidb-cloud"> | |
|
|
||
| > **Note:** | ||
| > | ||
| > Vector search index is only available for [TiDB Cloud Serverless](/tidb-cloud/select-cluster-tier.md#tidb-cloud-serverless) clusters. | ||
| > The vector search feature is only available for TiDB Self-Managed clusters and [TiDB Cloud Serverless](/tidb-cloud/select-cluster-tier.md#tidb-cloud-serverless) clusters. | ||
|
|
There was a problem hiding this comment.
Suggested change
| </CustomContent> | |
| - A distance function must be specified when creating and using vector search indexes (currently, only cosine distance `VEC_COSINE_DISTANCE()` and L2 distance `VEC_L2_DISTANCE()` functions are supported). | ||
| - For the same column, creating multiple vector search indexes using the same distance function is not supported. | ||
| - Deleting columns with vector search indexes is not supported. Creating multiple indexes in the same statement is not supported. | ||
| - Setting vector search indexes as [invisible](/sql-statements/sql-statement-alter-index.md) is not supported. |
There was a problem hiding this comment.
补上了这条：
- 不支持修改带有向量索引的列的类型（有损变更，即修改了列数据）。
Suggested change
| - Setting vector search indexes as [invisible](/sql-statements/sql-statement-alter-index.md) is not supported. | |
| - Modifying the type of a column with a vector index is not supported (lossy change, that is, column data is modified). | |
| - Setting vector search indexes as [invisible](/sql-statements/sql-statement-alter-index.md) is not supported. |
|
|
||
| ALTER TABLE foo ADD VECTOR INDEX idx_name ((VEC_COSINE_DISTANCE(data))) USING HNSW; | ||
| ``` | ||
|
|
There was a problem hiding this comment.
这注意建议删掉，在特性改为 GA 时很容易把此处漏改。L90 还有一处。
| @@ -95,15 +156,15 @@ SELECT * FROM | |||
| ) t | |||
| WHERE category = "document"; | |||
|
|
|||
| -- Note that this query may return less than 5 results if some are filtered out. | |||
| -- Note that this query might return less than 5 results if some are filtered out. | |||
There was a problem hiding this comment.
Suggested change
| -- Note that this query might return less than 5 results if some are filtered out. | |
| -- Note that this query might return fewer than 5 results if some are filtered out. |
| @@ -163,9 +251,11 @@ SELECT * FROM INFORMATION_SCHEMA.TIFLASH_INDEXES; | |||
|
|
|||
| For more information, see [`ALTER TABLE ... COMPACT`](/sql-statements/sql-statement-alter-table-compact.md). | |||
|
|
|||
| In addition, you can monitor the execution progress of the DDL job by executing `ADMIN SHOW DDL JOBS;` and checking the `row count`. However, this method is not fully accurate, because the `row count` value is obtained from the `rows_stable_indexed` field in `TIFLASH_INDEXES`. This approach can used as a reference for tracking the progress of indexing. | |||
There was a problem hiding this comment.
Suggested change
| In addition, you can monitor the execution progress of the DDL job by executing `ADMIN SHOW DDL JOBS;` and checking the `row count`. However, this method is not fully accurate, because the `row count` value is obtained from the `rows_stable_indexed` field in `TIFLASH_INDEXES`. This approach can used as a reference for tracking the progress of indexing. | |
| In addition, you can monitor the execution progress of the DDL job by executing `ADMIN SHOW DDL JOBS;` and checking the `row count`. However, this method is not fully accurate, because the `row count` value is obtained from the `rows_stable_indexed` field in `TIFLASH_INDEXES`. You can use this approach as a reference for tracking the progress of indexing. |
hfxsd
reviewed
Sep 30, 2024
|
|
||
| - `<HOST>`: The host of the TiDB cluster. | ||
| - `<PORT>`: The port of the TiDB cluster. | ||
| - `<USER>`: The username to connect to the TiDB cluster. |
hfxsd
reviewed
Sep 30, 2024
|
|
||
| - `<HOST>`: The host of the TiDB cluster. | ||
| - `<PORT>`: The port of the TiDB cluster. | ||
| - `<USER>`: The username to connect to the TiDB cluster. |
hfxsd
reviewed
Sep 30, 2024
|
|
||
| - `<HOST>`: The host of the TiDB cluster. | ||
| - `<PORT>`: The port of the TiDB cluster. | ||
| - `<USER>`: The username to connect to the TiDB cluster. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
First-time contributors' checklist
What is changed, added or deleted? (Required)
This PR moves 15 vector search docs from the tidb-cloud folder to the vector-search folder to so they can be reused by TiDB self-managed docs.
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?